{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Advanced Graphs\n",
    "\n",
    "Prequisites:\n",
    "\n",
    " - A running Kubernetes cluster\n",
    " - [Git clone of Seldon Core](https://github.com/SeldonIO/seldon-core)\n",
    " - [Helm](https://github.com/kubernetes/helm)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we will illustrate the different types of microservices that can be deployed in Seldon:\n",
    "* Model\n",
    "* Transformer\n",
    "* Router\n",
    "* Combiner\n",
    "* Output Transformer\n",
    "\n",
    "We will deploy graphs of increasing complexity. But first we need to install seldon on the cluster."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If running via Minikube you may need to ensure you have cluster-admin rights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['clusterrolebinding \"kube-system-cluster-admin\" created']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "!!kubectl create clusterrolebinding kube-system-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "serviceaccount \"tiller\" created\n",
      "clusterrolebinding \"tiller\" created\n",
      "$HELM_HOME has been configured at /home/clive/.helm.\n",
      "\n",
      "Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.\n",
      "\n",
      "Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.\n",
      "To prevent this, run `helm init` with the --tiller-tls-verify flag.\n",
      "For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation\n",
      "Happy Helming!\n"
     ]
    }
   ],
   "source": [
    "!kubectl -n kube-system create sa tiller\n",
    "!kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller\n",
    "!helm init --service-account tiller"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core-crd\n",
      "LAST DEPLOYED: Fri Sep  7 16:53:30 2018\n",
      "NAMESPACE: default\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ConfigMap\n",
      "NAME                     DATA  AGE\n",
      "seldon-spartakus-config  3     0s\n",
      "\n",
      "==> v1beta1/CustomResourceDefinition\n",
      "NAME                                         AGE\n",
      "seldondeployments.machinelearning.seldon.io  0s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                        DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-spartakus-volunteer  1        0        0           0          0s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME                        SECRETS  AGE\n",
      "seldon-spartakus-volunteer  1        0s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "==> v1beta1/ClusterRoleBinding\n",
      "NAME                        AGE\n",
      "seldon-spartakus-volunteer  0s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "NOTES: TODO\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core-crd --name seldon-core-crd --set usage_metrics.enabled=true"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "namespace \"graphs\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl create namespace graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NAME:   seldon-core\n",
      "LAST DEPLOYED: Fri Sep  7 17:05:59 2018\n",
      "NAMESPACE: graphs\n",
      "STATUS: DEPLOYED\n",
      "\n",
      "RESOURCES:\n",
      "==> v1/ClusterRoleBinding\n",
      "NAME    AGE\n",
      "seldon  1s\n",
      "\n",
      "==> v1beta1/Role\n",
      "NAME          AGE\n",
      "seldon-local  1s\n",
      "\n",
      "==> v1/RoleBinding\n",
      "NAME    AGE\n",
      "seldon  1s\n",
      "\n",
      "==> v1/Service\n",
      "NAME                          TYPE       CLUSTER-IP    EXTERNAL-IP  PORT(S)                        AGE\n",
      "seldon-core-seldon-apiserver  NodePort   10.110.68.62  <none>       8080:30024/TCP,5000:32281/TCP  1s\n",
      "seldon-core-redis             ClusterIP  10.101.74.51  <none>       6379/TCP                       1s\n",
      "\n",
      "==> v1beta1/Deployment\n",
      "NAME                                DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE\n",
      "seldon-core-seldon-apiserver        1        1        1           0          1s\n",
      "seldon-core-seldon-cluster-manager  1        1        1           0          1s\n",
      "seldon-core-redis                   1        1        1           0          1s\n",
      "\n",
      "==> v1/Pod(related)\n",
      "NAME                                                 READY  STATUS             RESTARTS  AGE\n",
      "seldon-core-seldon-apiserver-5949989877-47hxc        0/1    ContainerCreating  0         1s\n",
      "seldon-core-seldon-apiserver-5949989877-6v2ts        1/1    Terminating        0         12m\n",
      "seldon-core-seldon-cluster-manager-5594fbbcdc-pt9dn  0/1    ContainerCreating  0         1s\n",
      "seldon-core-redis-6f8d6bf976-fplp5                   0/1    ContainerCreating  0         1s\n",
      "\n",
      "==> v1/ServiceAccount\n",
      "NAME    SECRETS  AGE\n",
      "seldon  1        1s\n",
      "\n",
      "==> v1beta1/ClusterRole\n",
      "NAME        AGE\n",
      "seldon-crd  1s\n",
      "\n",
      "\n",
      "NOTES:\n",
      "Thank you for installing Seldon Core.\n",
      "\n",
      "Documentation can be found at https://github.com/SeldonIO/seldon-core\n",
      "\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "!helm install ../helm-charts/seldon-core --name seldon-core \\\n",
    "        --set cluster_manager.rbac=true \\\n",
    "        --namespace graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Set up REST and gRPC methods\n",
    "\n",
    "**Ensure you port forward the seldon api-server REST and GRPC ports**:\n",
    "\n",
    "REST:\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n graphs -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n graphs 8002:8080\n",
    "```\n",
    "\n",
    "GRPC:\n",
    "```\n",
    "kubectl port-forward $(kubectl get pods -n graphs -l app=seldon-apiserver-container-app -o jsonpath='{.items[0].metadata.name}') -n graphs 8003:5000\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cp ../proto/prediction.proto ./proto\n",
    "!python -m grpc.tools.protoc -I. --python_out=. --grpc_python_out=. ./proto/prediction.proto"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from visualizer import get_graph\n",
    "import json\n",
    "from seldon_utils import *\n",
    "API_GATEWAY_REST=\"localhost:8002\"\n",
    "API_GATEWAY_GRPC=\"localhost:8003\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"106pt\" height=\"171pt\"\n",
       " viewBox=\"0.00 0.00 106.00 171.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 167)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-167 102,-167 102,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-155 90,-155 90,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"49\" y=\"-139.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<!-- classifier0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>classifier0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"81.5,-124 16.5,-124 16.5,-88 81.5,-88 81.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"49\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier</text>\n",
       "</g>\n",
       "<!-- classifier0endpoint -->\n",
       "<g id=\"node2\" class=\"node\"><title>classifier0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"49\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"49\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier0&#45;&gt;classifier0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>classifier0&#45;&gt;classifier0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M49,-87.6966C49,-79.9827 49,-70.7125 49,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"52.5001,-62.1043 49,-52.1043 45.5001,-62.1044 52.5001,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fb5684e0b00>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/model.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we will check that everything works by running a simple model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-model\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/model.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-example-svc-orch replicas:1 replicasAvailable:1] map[replicas:1 replicasAvailable:1 name:test-deployment-example-classifier-0]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-model -o jsonpath=\"{.status}\" -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":43199,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"451qsqv560121tt9skumrg0top\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07829565259771333]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":43179,\"scope\":\"read write\"}\n",
      "meta {\n",
      "  puid: \"4jlaasifq85i9cvc6s44pnorln\"\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 1\n",
      "    values: 0.09835821473688332\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-model\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/model.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Random AB Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"213pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 213.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-239 209,-239 209,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-227 197,-227 197,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"102.5\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<!-- random&#45;ab&#45;test0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>random&#45;ab&#45;test0</title>\n",
       "<polygon fill=\"lightgrey\" stroke=\"lightgrey\" points=\"151.5,-196 52.5,-196 52.5,-160 151.5,-160 151.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"102\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">random&#45;ab&#45;test</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"93.5,-124 16.5,-124 16.5,-88 93.5,-88 93.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"55\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- random&#45;ab&#45;test0&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>random&#45;ab&#45;test0&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M90.382,-159.697C84.8612,-151.474 78.153,-141.483 72.0682,-132.421\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"74.9645,-130.456 66.4843,-124.104 69.1529,-134.358 74.9645,-130.456\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node4\" class=\"node\"><title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"188.5,-124 111.5,-124 111.5,-88 188.5,-88 188.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"150\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- random&#45;ab&#45;test0&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>random&#45;ab&#45;test0&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M113.865,-159.697C119.503,-151.474 126.354,-141.483 132.569,-132.421\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"135.503,-134.331 138.271,-124.104 129.729,-130.372 135.503,-134.331\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node3\" class=\"node\"><title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"55\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"55\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M55,-87.6966C55,-79.9827 55,-70.7125 55,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"58.5001,-62.1043 55,-52.1043 51.5001,-62.1044 58.5001,-62.1043\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node5\" class=\"node\"><title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"150\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"150\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M150,-87.6966C150,-79.9827 150,-70.7125 150,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"153.5,-62.1043 150,-52.1043 146.5,-62.1044 153.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fb5447a2278>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/random_ab_test.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we will deploy 2 models under an AB test router. The Random AB Test we will use is implemented directly in Seldon, this is not a microservice, so no docker image needs to be specified.\n",
    "\n",
    "The json graph is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'children': [{'children': [],\n",
       "   'endpoint': {'type': 'REST'},\n",
       "   'name': 'classifier-1',\n",
       "   'type': 'MODEL'},\n",
       "  {'children': [],\n",
       "   'endpoint': {'type': 'REST'},\n",
       "   'name': 'classifier-2',\n",
       "   'type': 'MODEL'}],\n",
       " 'endpoint': {},\n",
       " 'implementation': 'RANDOM_ABTEST',\n",
       " 'name': 'random-ab-test',\n",
       " 'parameters': [{'name': 'ratioA', 'type': 'FLOAT', 'value': '0.5'}]}"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/random_ab_test.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We specify ``` \"implementation\": \"RANDOM_ABTEST\" ``` to get the AB Test router implemented in Seldon.\n",
    "We pass the parameter ratioA which corresponds to the ratio of requests that will be passed to the first child of the Router"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/random_ab_test.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-abtest-abtest-svc-orch replicas:1 replicasAvailable:1] map[name:test-deployment-abtest-abtest-classifier-1-0 replicas:1 replicasAvailable:1] map[name:test-deployment-abtest-abtest-classifier-2-1 replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":43047,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"bvhqfc5mc19ubfm5sjv2hotv1e\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"random-ab-test\": 1\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.0869949501192126]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":43033,\"scope\":\"read write\"}\n",
      "meta {\n",
      "  puid: \"4v9kklilc2nnrhpjurbevnn3ni\"\n",
      "  routing {\n",
      "    key: \"random-ab-test\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 1\n",
      "    values: 0.08726940781776818\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/random_ab_test.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Average Combiner"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"308pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 308.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-239 304,-239 304,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-227 292,-227 292,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"150\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<!-- ensemble0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>ensemble0</title>\n",
       "<polygon fill=\"lightgrey\" stroke=\"lightgrey\" points=\"184,-196 116,-196 116,-160 184,-160 184,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"150\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">ensemble</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node2\" class=\"node\"><title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"93.5,-124 16.5,-124 16.5,-88 93.5,-88 93.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"55\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- ensemble0&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>ensemble0&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M126.517,-159.697C114.437,-150.796 99.5458,-139.823 86.4842,-130.199\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"88.3398,-127.219 78.213,-124.104 84.1874,-132.854 88.3398,-127.219\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node4\" class=\"node\"><title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"188.5,-124 111.5,-124 111.5,-88 188.5,-88 188.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"150\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- ensemble0&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>ensemble0&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M150,-159.697C150,-151.983 150,-142.712 150,-134.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"153.5,-134.104 150,-124.104 146.5,-134.104 153.5,-134.104\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3 -->\n",
       "<g id=\"node6\" class=\"node\"><title>classifier&#45;3</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"283.5,-124 206.5,-124 206.5,-88 283.5,-88 283.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"245\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;3</text>\n",
       "</g>\n",
       "<!-- ensemble0&#45;&gt;classifier&#45;3 -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>ensemble0&#45;&gt;classifier&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M173.483,-159.697C185.563,-150.796 200.454,-139.823 213.516,-130.199\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"215.813,-132.854 221.787,-124.104 211.66,-127.219 215.813,-132.854\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node3\" class=\"node\"><title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"55\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"55\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M55,-87.6966C55,-79.9827 55,-70.7125 55,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"58.5001,-62.1043 55,-52.1043 51.5001,-62.1044 58.5001,-62.1043\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node5\" class=\"node\"><title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"150\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"150\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M150,-87.6966C150,-79.9827 150,-70.7125 150,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"153.5,-62.1043 150,-52.1043 146.5,-62.1044 153.5,-62.1043\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3endpoint -->\n",
       "<g id=\"node7\" class=\"node\"><title>classifier&#45;3endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"245\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"245\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;3&#45;&gt;classifier&#45;3endpoint -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>classifier&#45;3&#45;&gt;classifier&#45;3endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M245,-87.6966C245,-79.9827 245,-70.7125 245,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"248.5,-62.1043 245,-52.1043 241.5,-62.1044 248.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fb54474a9b0>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/ensemble.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example again we will use a service implemented in seldon, called Average Combiner. It takes the outputs of several models and returns the arithmetic mean of them.\n",
    "\n",
    "The json is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'children': [{'endpoint': {'type': 'REST'},\n",
       "   'name': 'classifier-1',\n",
       "   'type': 'MODEL'},\n",
       "  {'endpoint': {'type': 'REST'}, 'name': 'classifier-2', 'type': 'MODEL'},\n",
       "  {'endpoint': {'type': 'REST'}, 'name': 'classifier-3', 'type': 'MODEL'}],\n",
       " 'implementation': 'AVERAGE_COMBINER',\n",
       " 'name': 'ensemble'}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/ensemble.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/ensemble.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-ensembles-ensemble-classifier-1-0 replicas:1 replicasAvailable:1] map[name:test-deployment-ensembles-ensemble-svc-orch replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":42817,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"status\": {\n",
      "    \"code\": 0,\n",
      "    \"info\": \"\",\n",
      "    \"reason\": \"\",\n",
      "    \"status\": \"SUCCESS\"\n",
      "  },\n",
      "  \"meta\": {\n",
      "    \"puid\": \"6brqopn2kgj2mei19v62u1s1o2\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"ensemble\": -1\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.06432655349966468]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":42804,\"scope\":\"read write\"}\n",
      "status {\n",
      "}\n",
      "meta {\n",
      "  puid: \"4qdusjsnvfelfapekc4a41n048\"\n",
      "  routing {\n",
      "    key: \"ensemble\"\n",
      "    value: -1\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 1\n",
      "    values: 0.09087465642287341\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/ensemble.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feature Transformer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"188pt\" height=\"243pt\"\n",
       " viewBox=\"0.00 0.00 188.00 243.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 239)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-239 184,-239 184,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-227 172,-227 172,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"90\" y=\"-211.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<!-- transformer0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>transformer0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"130,-196 50,-196 50,-160 130,-160 130,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"90\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">transformer</text>\n",
       "</g>\n",
       "<!-- transformer0endpoint -->\n",
       "<g id=\"node2\" class=\"node\"><title>transformer0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"48\" cy=\"-106\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"48\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- transformer0&#45;&gt;transformer0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>transformer0&#45;&gt;transformer0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M79.618,-159.697C74.6624,-151.437 68.6364,-141.394 63.1797,-132.299\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"66.1196,-130.396 57.9734,-123.622 60.1172,-133.998 66.1196,-130.396\"/>\n",
       "</g>\n",
       "<!-- classifier -->\n",
       "<g id=\"node3\" class=\"node\"><title>classifier</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"163.5,-124 98.5,-124 98.5,-88 163.5,-88 163.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"131\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier</text>\n",
       "</g>\n",
       "<!-- transformer0&#45;&gt;classifier -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>transformer0&#45;&gt;classifier</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M100.135,-159.697C104.852,-151.644 110.562,-141.894 115.782,-132.982\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"118.948,-134.502 120.982,-124.104 112.908,-130.964 118.948,-134.502\"/>\n",
       "</g>\n",
       "<!-- classifierendpoint -->\n",
       "<g id=\"node4\" class=\"node\"><title>classifierendpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"131\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"131\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;&gt;classifierendpoint -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>classifier&#45;&gt;classifierendpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M131,-87.6966C131,-79.9827 131,-70.7125 131,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"134.5,-62.1043 131,-52.1043 127.5,-62.1044 134.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fb54474aa90>"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/feature_transform.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we deploy a simple model under a feature transformation microservice. For the transformer we will use the docker image seldonio/mock_transformer:1.0\n",
    "\n",
    "Since this is not implemented in Seldon, the type of predictive unit (TRANSFORMER) needs to be specified in the graph so that Seldon Core knows which API this microservice implements.\n",
    "\n",
    "The json is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'children': [{'endpoint': {'type': 'REST'},\n",
       "   'name': 'classifier',\n",
       "   'type': 'MODEL'}],\n",
       " 'endpoint': {'type': 'REST'},\n",
       " 'name': 'transformer',\n",
       " 'type': 'TRANSFORMER'}"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/feature_transform.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/feature_transform.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[replicasAvailable:1 name:test-deployment-transformer-transformer-svc-orch replicas:1] map[name:test-deployment-transformer-transformer-classifier-0 replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":42676,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"609lofmkg4lsulscaaenoh1igs\",\n",
      "    \"tags\": {\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"transformer\": -1\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [1, 1],\n",
      "      \"values\": [0.07725291332801949]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":42660,\"scope\":\"read write\"}\n",
      "meta {\n",
      "  puid: \"1lofl8bsm4udb5ch15cuvgr4c9\"\n",
      "  routing {\n",
      "    key: \"transformer\"\n",
      "    value: -1\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 1\n",
      "    shape: 1\n",
      "    values: 0.08068693474059799\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/feature_transform.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Outlier Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"218pt\" height=\"315pt\"\n",
       " viewBox=\"0.00 0.00 218.00 315.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 311)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-311 214,-311 214,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-299 202,-299 202,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"105\" y=\"-283.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>outlier&#45;detector0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"155,-268 55,-268 55,-232 155,-232 155,-268\"/>\n",
       "<text text-anchor=\"middle\" x=\"105\" y=\"-246.3\" font-family=\"Times,serif\" font-size=\"14.00\">outlier&#45;detector</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0endpoint -->\n",
       "<g id=\"node2\" class=\"node\"><title>outlier&#45;detector0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"48\" cy=\"-178\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"48\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M90.9101,-231.697C83.7706,-222.929 74.9938,-212.15 67.2421,-202.631\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"69.7892,-200.216 60.7609,-194.671 64.3611,-204.636 69.7892,-200.216\"/>\n",
       "</g>\n",
       "<!-- random&#45;abtest -->\n",
       "<g id=\"node3\" class=\"node\"><title>random&#45;abtest</title>\n",
       "<polygon fill=\"lightgrey\" stroke=\"lightgrey\" points=\"193.5,-196 98.5,-196 98.5,-160 193.5,-160 193.5,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"146\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">random&#45;abtest</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;random&#45;abtest -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>outlier&#45;detector0&#45;&gt;random&#45;abtest</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M115.135,-231.697C119.852,-223.644 125.562,-213.894 130.782,-204.982\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"133.948,-206.502 135.982,-196.104 127.908,-202.964 133.948,-206.502\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node4\" class=\"node\"><title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"95.5,-124 18.5,-124 18.5,-88 95.5,-88 95.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"57\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>random&#45;abtest&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M124,-159.697C112.791,-150.881 98.9979,-140.032 86.8458,-130.474\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"88.7708,-127.535 78.7469,-124.104 84.4433,-133.038 88.7708,-127.535\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node6\" class=\"node\"><title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"190.5,-124 113.5,-124 113.5,-88 190.5,-88 190.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"152\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>random&#45;abtest&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M147.483,-159.697C148.144,-151.983 148.939,-142.712 149.676,-134.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"153.167,-134.367 150.534,-124.104 146.193,-133.769 153.167,-134.367\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node5\" class=\"node\"><title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"57\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"57\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M57,-87.6966C57,-79.9827 57,-70.7125 57,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"60.5001,-62.1043 57,-52.1043 53.5001,-62.1044 60.5001,-62.1043\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node7\" class=\"node\"><title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"152\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"152\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M152,-87.6966C152,-79.9827 152,-70.7125 152,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"155.5,-62.1043 152,-52.1043 148.5,-62.1044 155.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7fb5447a24e0>"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/outlier_detector.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we will have different four components: \n",
    "* A transformer, the outlier detector\n",
    "* A router, the random AB test\n",
    "* Two models\n",
    "\n",
    "The outlier detector is a special kind of transformer that will populate a tag in the response metadata with the outlier score it has calculated. \n",
    "We use the docker image seldonio/outlier_mahalanobis:0.2 for the outlier detector.\n",
    "\n",
    "The json is as follows:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'children': [{'children': [{'endpoint': {'type': 'REST'},\n",
       "     'name': 'classifier-1',\n",
       "     'type': 'MODEL'},\n",
       "    {'endpoint': {'type': 'REST'}, 'name': 'classifier-2', 'type': 'MODEL'}],\n",
       "   'implementation': 'RANDOM_ABTEST',\n",
       "   'name': 'random-abtest',\n",
       "   'parameters': [{'name': 'ratioA', 'type': 'FLOAT', 'value': '0.5'}],\n",
       "   'type': 'UNKNOWN_TYPE'}],\n",
       " 'endpoint': {'type': 'REST'},\n",
       " 'name': 'outlier-detector',\n",
       " 'type': 'TRANSFORMER'}"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json.load(open(\"./resources/outlier_detector.json\",'r')).get(\"spec\").get(\"predictors\")[0].get(\"graph\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/outlier_detector.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-complex-outlier-svc-orch replicas:1 replicasAvailable:1] map[name:test-deployment-complex-outlier-classifier-1-0 replicas:1 replicasAvailable:1] map[name:test-deployment-complex-outlier-outlier-detector-1 replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":41909,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"meta\": {\n",
      "    \"puid\": \"t4rbm3rk6b39ik542tn0tq91ns\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": [0.0, 0.0]\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1,\n",
      "      \"random-abtest\": 1\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [2, 1],\n",
      "      \"values\": [0.08451014845764081, 0.08581033201511856]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":41896,\"scope\":\"read write\"}\n",
      "meta {\n",
      "  puid: \"p4vvcd4a78v7ij3d3hvjbc9i0n\"\n",
      "  tags {\n",
      "    key: \"outlierScore\"\n",
      "    value {\n",
      "      list_value {\n",
      "        values {\n",
      "          number_value: 0.0\n",
      "        }\n",
      "        values {\n",
      "          number_value: 0.0\n",
      "        }\n",
      "      }\n",
      "    }\n",
      "  }\n",
      "  routing {\n",
      "    key: \"outlier-detector\"\n",
      "    value: -1\n",
      "  }\n",
      "  routing {\n",
      "    key: \"random-abtest\"\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 2\n",
      "    shape: 1\n",
      "    values: 0.07959244469471599\n",
      "    values: 0.07167298063671615\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/outlier_detector.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complex Graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\n",
       "<!DOCTYPE svg PUBLIC \"-//W3C//DTD SVG 1.1//EN\"\n",
       " \"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd\">\n",
       "<!-- Generated by graphviz version 2.38.0 (20140413.2041)\n",
       " -->\n",
       "<!-- Title: %3 Pages: 1 -->\n",
       "<svg width=\"390pt\" height=\"387pt\"\n",
       " viewBox=\"0.00 0.00 390.00 387.00\" xmlns=\"http://www.w3.org/2000/svg\" xmlns:xlink=\"http://www.w3.org/1999/xlink\">\n",
       "<g id=\"graph0\" class=\"graph\" transform=\"scale(1 1) rotate(0) translate(4 383)\">\n",
       "<title>%3</title>\n",
       "<polygon fill=\"white\" stroke=\"none\" points=\"-4,4 -4,-383 386,-383 386,4 -4,4\"/>\n",
       "<g id=\"clust1\" class=\"cluster\"><title>cluster_0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"8,-8 8,-371 374,-371 374,-8 8,-8\"/>\n",
       "<text text-anchor=\"middle\" x=\"191\" y=\"-355.8\" font-family=\"Times,serif\" font-size=\"14.00\">predictor</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0 -->\n",
       "<g id=\"node1\" class=\"node\"><title>outlier&#45;detector0</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"156,-340 56,-340 56,-304 156,-304 156,-340\"/>\n",
       "<text text-anchor=\"middle\" x=\"106\" y=\"-318.3\" font-family=\"Times,serif\" font-size=\"14.00\">outlier&#45;detector</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0endpoint -->\n",
       "<g id=\"node2\" class=\"node\"><title>outlier&#45;detector0endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"77\" cy=\"-250\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"77\" y=\"-246.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint -->\n",
       "<g id=\"edge1\" class=\"edge\"><title>outlier&#45;detector0&#45;&gt;outlier&#45;detector0endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M98.8315,-303.697C95.4811,-295.609 91.422,-285.812 87.7172,-276.869\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"90.9473,-275.521 83.8864,-267.622 84.4803,-278.2 90.9473,-275.521\"/>\n",
       "</g>\n",
       "<!-- random&#45;abtest -->\n",
       "<g id=\"node3\" class=\"node\"><title>random&#45;abtest</title>\n",
       "<polygon fill=\"lightgrey\" stroke=\"lightgrey\" points=\"222.5,-268 127.5,-268 127.5,-232 222.5,-232 222.5,-268\"/>\n",
       "<text text-anchor=\"middle\" x=\"175\" y=\"-246.3\" font-family=\"Times,serif\" font-size=\"14.00\">random&#45;abtest</text>\n",
       "</g>\n",
       "<!-- outlier&#45;detector0&#45;&gt;random&#45;abtest -->\n",
       "<g id=\"edge11\" class=\"edge\"><title>outlier&#45;detector0&#45;&gt;random&#45;abtest</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M123.056,-303.697C131.495,-295.135 141.825,-284.656 151.043,-275.304\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"153.613,-277.683 158.14,-268.104 148.627,-272.769 153.613,-277.683\"/>\n",
       "</g>\n",
       "<!-- transformer -->\n",
       "<g id=\"node4\" class=\"node\"><title>transformer</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"177,-196 97,-196 97,-160 177,-160 177,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"137\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">transformer</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;transformer -->\n",
       "<g id=\"edge5\" class=\"edge\"><title>random&#45;abtest&#45;&gt;transformer</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M165.607,-231.697C161.235,-223.644 155.943,-213.894 151.105,-204.982\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"154.132,-203.223 146.285,-196.104 147.98,-206.563 154.132,-203.223\"/>\n",
       "</g>\n",
       "<!-- ensemble -->\n",
       "<g id=\"node8\" class=\"node\"><title>ensemble</title>\n",
       "<polygon fill=\"lightgrey\" stroke=\"lightgrey\" points=\"266,-196 198,-196 198,-160 266,-160 266,-196\"/>\n",
       "<text text-anchor=\"middle\" x=\"232\" y=\"-174.3\" font-family=\"Times,serif\" font-size=\"14.00\">ensemble</text>\n",
       "</g>\n",
       "<!-- random&#45;abtest&#45;&gt;ensemble -->\n",
       "<g id=\"edge10\" class=\"edge\"><title>random&#45;abtest&#45;&gt;ensemble</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M189.09,-231.697C195.923,-223.305 204.257,-213.07 211.756,-203.861\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"214.472,-206.069 218.072,-196.104 209.044,-201.649 214.472,-206.069\"/>\n",
       "</g>\n",
       "<!-- transformerendpoint -->\n",
       "<g id=\"node5\" class=\"node\"><title>transformerendpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"48\" cy=\"-106\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"48\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- transformer&#45;&gt;transformerendpoint -->\n",
       "<g id=\"edge2\" class=\"edge\"><title>transformer&#45;&gt;transformerendpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M115,-159.697C102.586,-149.932 87.0007,-137.675 73.9867,-127.439\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"75.8889,-124.482 65.865,-121.051 71.5614,-129.984 75.8889,-124.482\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1 -->\n",
       "<g id=\"node6\" class=\"node\"><title>classifier&#45;1</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"175.5,-124 98.5,-124 98.5,-88 175.5,-88 175.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"137\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;1</text>\n",
       "</g>\n",
       "<!-- transformer&#45;&gt;classifier&#45;1 -->\n",
       "<g id=\"edge4\" class=\"edge\"><title>transformer&#45;&gt;classifier&#45;1</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M137,-159.697C137,-151.983 137,-142.712 137,-134.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"140.5,-134.104 137,-124.104 133.5,-134.104 140.5,-134.104\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;1endpoint -->\n",
       "<g id=\"node7\" class=\"node\"><title>classifier&#45;1endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"137\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"137\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;1&#45;&gt;classifier&#45;1endpoint -->\n",
       "<g id=\"edge3\" class=\"edge\"><title>classifier&#45;1&#45;&gt;classifier&#45;1endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M137,-87.6966C137,-79.9827 137,-70.7125 137,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"140.5,-62.1043 137,-52.1043 133.5,-62.1044 140.5,-62.1043\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2 -->\n",
       "<g id=\"node9\" class=\"node\"><title>classifier&#45;2</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"270.5,-124 193.5,-124 193.5,-88 270.5,-88 270.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"232\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;2</text>\n",
       "</g>\n",
       "<!-- ensemble&#45;&gt;classifier&#45;2 -->\n",
       "<g id=\"edge7\" class=\"edge\"><title>ensemble&#45;&gt;classifier&#45;2</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M232,-159.697C232,-151.983 232,-142.712 232,-134.112\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"235.5,-134.104 232,-124.104 228.5,-134.104 235.5,-134.104\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3 -->\n",
       "<g id=\"node11\" class=\"node\"><title>classifier&#45;3</title>\n",
       "<polygon fill=\"none\" stroke=\"black\" points=\"365.5,-124 288.5,-124 288.5,-88 365.5,-88 365.5,-124\"/>\n",
       "<text text-anchor=\"middle\" x=\"327\" y=\"-102.3\" font-family=\"Times,serif\" font-size=\"14.00\">classifier&#45;3</text>\n",
       "</g>\n",
       "<!-- ensemble&#45;&gt;classifier&#45;3 -->\n",
       "<g id=\"edge9\" class=\"edge\"><title>ensemble&#45;&gt;classifier&#45;3</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M255.483,-159.697C267.563,-150.796 282.454,-139.823 295.516,-130.199\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"297.813,-132.854 303.787,-124.104 293.66,-127.219 297.813,-132.854\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;2endpoint -->\n",
       "<g id=\"node10\" class=\"node\"><title>classifier&#45;2endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"232\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"232\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;2&#45;&gt;classifier&#45;2endpoint -->\n",
       "<g id=\"edge6\" class=\"edge\"><title>classifier&#45;2&#45;&gt;classifier&#45;2endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M232,-87.6966C232,-79.9827 232,-70.7125 232,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"235.5,-62.1043 232,-52.1043 228.5,-62.1044 235.5,-62.1043\"/>\n",
       "</g>\n",
       "<!-- classifier&#45;3endpoint -->\n",
       "<g id=\"node12\" class=\"node\"><title>classifier&#45;3endpoint</title>\n",
       "<ellipse fill=\"none\" stroke=\"black\" cx=\"327\" cy=\"-34\" rx=\"32.4942\" ry=\"18\"/>\n",
       "<text text-anchor=\"middle\" x=\"327\" y=\"-30.3\" font-family=\"Times,serif\" font-size=\"14.00\">REST</text>\n",
       "</g>\n",
       "<!-- classifier&#45;3&#45;&gt;classifier&#45;3endpoint -->\n",
       "<g id=\"edge8\" class=\"edge\"><title>classifier&#45;3&#45;&gt;classifier&#45;3endpoint</title>\n",
       "<path fill=\"none\" stroke=\"black\" d=\"M327,-87.6966C327,-79.9827 327,-70.7125 327,-62.1124\"/>\n",
       "<polygon fill=\"black\" stroke=\"black\" points=\"330.5,-62.1043 327,-52.1043 323.5,-62.1044 330.5,-62.1043\"/>\n",
       "</g>\n",
       "</g>\n",
       "</svg>\n"
      ],
      "text/plain": [
       "<graphviz.dot.Digraph at 0x7faccf85bf60>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_graph(\"resources/complex_graph.json\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this final example we will deploy a complex graph with all of the components that have been used so far."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" created\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl apply -f resources/complex_graph.json -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "map[predictorStatus:[map[name:test-deployment-complex-complex-svc-orch replicas:1 replicasAvailable:1] map[replicasAvailable:1 name:test-deployment-complex-complex-classifier-1-0 replicas:1] map[name:test-deployment-complex-complex-classifier-2-1 replicas:1 replicasAvailable:1] map[name:test-deployment-complex-complex-classifier-3-2 replicas:1 replicasAvailable:1] map[name:test-deployment-complex-complex-transformer-4 replicas:1 replicasAvailable:1] map[name:test-deployment-complex-complex-outlier-detector-3 replicas:1 replicasAvailable:1]]]"
     ]
    }
   ],
   "source": [
    "!kubectl get seldondeployments seldon-deployment -o jsonpath='{.status}' -n graphs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":41760,\"scope\":\"read write\"}\n",
      "{\n",
      "  \"status\": {\n",
      "    \"code\": 0,\n",
      "    \"info\": \"\",\n",
      "    \"reason\": \"\",\n",
      "    \"status\": \"SUCCESS\"\n",
      "  },\n",
      "  \"meta\": {\n",
      "    \"puid\": \"o5l2cpfqjijoaa4ormjbskipql\",\n",
      "    \"tags\": {\n",
      "      \"outlierScore\": 0.9008780576448335\n",
      "    },\n",
      "    \"routing\": {\n",
      "      \"outlier-detector\": -1,\n",
      "      \"ensemble\": -1,\n",
      "      \"random-abtest\": 1\n",
      "    }\n",
      "  },\n",
      "  \"data\": {\n",
      "    \"names\": [\"proba\"],\n",
      "    \"tensor\": {\n",
      "      \"shape\": [2, 1],\n",
      "      \"values\": [0.09485971576799879, 0.08104735785381909]\n",
      "    }\n",
      "  }\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "rest_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\"access_token\":\"de4efb0c-0166-4d95-b1e6-bd096748842d\",\"token_type\":\"bearer\",\"expires_in\":41747,\"scope\":\"read write\"}\n",
      "meta {\n",
      "  puid: \"iffpdo8ouq2ehpqbhluav89g6a\"\n",
      "  tags {\n",
      "    key: \"outlierScore\"\n",
      "    value {\n",
      "      number_value: 0.5414579901687784\n",
      "    }\n",
      "  }\n",
      "  routing {\n",
      "    key: \"outlier-detector\"\n",
      "    value: -1\n",
      "  }\n",
      "  routing {\n",
      "    key: \"random-abtest\"\n",
      "  }\n",
      "  routing {\n",
      "    key: \"transformer\"\n",
      "    value: -1\n",
      "  }\n",
      "}\n",
      "data {\n",
      "  names: \"proba\"\n",
      "  tensor {\n",
      "    shape: 2\n",
      "    shape: 1\n",
      "    values: 0.08611808920667463\n",
      "    values: 0.08029119284002609\n",
      "  }\n",
      "}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "grpc_request_api_gateway(\"oauth-key\",\"oauth-secret\",API_GATEWAY_REST,API_GATEWAY_GRPC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "seldondeployment \"seldon-deployment\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!kubectl delete -f resources/complex_graph.json -n graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Tear Down"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"seldon-core\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete seldon-core --purge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "release \"seldon-core-crd\" deleted\r\n"
     ]
    }
   ],
   "source": [
    "!helm delete seldon-core-crd --purge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
